Measuring Human Intelligence with Artificial Intelligence Adaptive Item Generation

Adaptive item generation may be the next innovation in intelligence testing. In adaptive item generation, the optimally informative item is developed anew for the examinee during the test. Reminiscent of computer versus person chess games, the computer generates the next item based on the previous pattern of the examinee's responses. Adaptive item generation requires the merger of two lines of research, psychometric methods for adaptive testing and a cognitive analysis of items. Adaptive testing is the current state of the art in intelligence measurement. In adaptive testing, items are selected individually for optimal information about an examinee's ability during testing. The items are selected interactively by a computer algorithm using calibrated psychometric properties. Generally, harder items are selected if the examinee solves items, while easier ones are selected if the examinee does not solve items. Adaptive item selection leads to shorter and more reliable tests. In a sense, optimal item selection for an examinee is measurement by artificial intelligence. Adaptive item generation is a step beyond adaptive testing. Like adaptive testing, it estimates the psychometric properties of the optimally informative items for the person. Beyond this, however, the impact of specific stimulus content on an item's psychometric properties must be known. That is, knowledge is required of how stimulus features in specific items impact the ability construct. This chapter describes a system for measuring ability in which new items are created while the person takes the test. Ability is measured online by a system of artificial intelligence. The items that are created are designed to be optimally informative about the person's ability. The system behind the item generation is the cognitive design system approach (Embretson, 1998).